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Aerospace systems are developed similarly to other large-scale systems through a series 
of reviews, where designs are modified as system requirements are refined. For space-based 
systems few are built and placed into service — these research vehicles have limited historical 
experience to draw from and formidable reliability and safety requirements, due to the 
remote and severe environment of space. Aeronautical systems have similar reliability and 
safety requirements, and while these systems may have historical information to access, 
commercial and military systems require longevity under a range of operational conditions 
and applied loads. Historically, the design of aerospace systems, particularly the selection of 
sensors, is based on the requirements for control and performance rather than on health 
assessment needs. Furthermore, the safety and reliability requirements are met through 
sensor suite augmentation in an ad hoc, heuristic manner, rather than any systematic 
approach. A review of the current sensor selection practice within and outside of the 
aerospace community was conducted and a sensor selection architecture is proposed that 
will provide a justifiable, defendable sensor suite to address system health assessment 
requirements. 
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Nomenclature 


A-j jq = influence coefficient for hardware fault parameter j and measurement i at interpolation 

point p ]q 

Ck = criticality weighting factor of fault test case k 

D residual j = residual measurement agreement metric for hardware fault parameter j 

dj = closest point along fault trajectory for fault scenario j to the current observation 

f = effective engine simulation model functional relation for sensor i 

FPOV = Fuel Preburned Oxidizer Value 

HPFP = High Pressure Fuel Pump 

HPFT = High Pressure Fuel Turbine 

F1POP = High Pressure Oxidizer Pump 

F1POT = High Pressure Oxidizer Turbine 

K = normalizing term for the penalty function 

k = number of fault scenarios 

LPFP = Low Pressure Fuel Pump 

LPFT = Low Pressure Fuel Turbine 

LPOP = Low Pressure Oxidizer Pump 

LPOT = Low Pressure Oxidizer Turbine 

m = number of measurements in the sensor suite 

MCC = Main Combustion Chamber 

N desired = desired number of sensors in solution sensor suite 

Nactuai = actual number of sensors in candidate sensor suite 

n = number of hardware fault parameters/fault scenarios 

OPOV = Oxidizer Preburner Oxidizer Value 

P = penalty weighting term of the sensor suite 

q = number of fault test cases 

s = number of interpolation points for each hardware fault parameter 

Tj = threshold of measurement residual agreement for measurement i 

2 

American Institute of Aeronautics and Astronautics 



u, 

u 


w k 


w, 


penalty 


X conditioned , j 
X baseline, j 



y baseline, i 
y observed ,i 
y conditioned ,i 


yP » 


yt 

y resid _ err, i 
y residual, i 

z k 


utility cost term applied to each sensor in the current suite 

utility weighting term of the current sensor suite 

addition weighting factor for fault test case k 

weight of the penalty term 

predicted value of hardware fault parameter j 

value of hardware fault parameter j 
conditioned value of hardware fault parameter j 
nominal baseline value of hardware fault parameter / 

value of hardware fault parameter j at interpolation point p Jq 
nominal value for measurement i 

observed value of measurement i at the specified fault state 
conditioned value for measurement i at the specified fault state 

predicted value of measurement y t when x f = x p , and all other dimensionless hardware 

fault parameters remain at their nominal value 
predicted value of measurement i 
error in residual value of measurement i 
recovered residual value of measurement ; 
fault discrimination metric for fault test case k 
in-run uncertainty characteristic of measurement i 


I. Introduction 

. EROSPACE systems are developed similarly to other large-scale systems through a series of design reviews. 

A ^ 

At each design stage, decisions and trades are made about all aspects of the vehicle: hardware, software, 
functionality, operations, etc. Each component, including sensors, must “buy its way on board.” The costs and 
risks associated with flight dictate that unnecessary components be filtered from the design; therefore all 
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components must justify their existence and purpose onboard the vehicle. To achieve this, a component must satisfy 
a quantifiable and verifiable set of requirements that will demonstrate its benefit and purpose to the system. In 
addition, any risk to the system must also be identified. 

Traditionally, in flight systems, sensors are primarily selected through an ad hoc heuristic process, where various 
domain groups are polled as to what sensors they require in the system. Although safety and reliability are just as 
important as system performance, the sensors are primarily selected based on control requirements and performance 
assessment, rather than on health monitoring and management. To be incorporated into the design process, accepted 
methodologies and procedures must be established that provide justification of the selected sensors within the 
constraints of the vehicle requirements. To precisely quantify the benefits of sensors in flight systems, heuristic 
sensor selection approaches must give way to systematic techniques that are based on accepted selection criteria and 
performance metrics that measure their value to the system. 

Therefore, substantial benefit to the design and development process can be realized by utilizing a sensor 
selection methodology, specifically directed toward system health assessment. The purpose of this study is to review 
the current state of sensor selection within and outside of the aerospace system community and to propose a sensor 
selection architecture that will provide justification of sensor inclusion based on the net benefit from the health 
assessment perspective. This review is not intended to be encompassing in terms of directly comparing the various 
techniques against each other and assessing their performance. Rather, the review is based on categorizing salient 
features of these approaches and the desirability of certain features when they are applied to aerospace flight 
systems. 

The paper is organized as follows. Sections II and III will review and discuss various techniques used 
historically for sensor selection. Section IV will discuss in detail the proposed sensor selection architecture 
originally designed to address health assessment for propulsion systems with a discussion of proposed research areas 
that will advance the capabilities and credibility of the sensor selection process for aerospace diagnostic 
applications. In Section V, a design example will demonstrate an application of the proposed sensor selection 
technique. 
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II. An Overview of the Sensor Selection Process 

A number of sensor selection approaches, applied to a variety of systems and for a variety of purposes, have 
been developed over the years. For the methods described in this review, the sensor selection process can be divided 
into two parts: an evaluation module and an optimization module. The evaluation module generates a performance 
metric for the sensor suite or network under review, and the optimization module searches through the space of all 
possible sensor suites and selects additional suites for further evaluation. The process is continued until the “best” or 
the optimum sensor suite is obtained. 

A. Sensor Selection Evaluation 

The goal of the sensor selection process is to provide a suite of sensors that fulfill specified performance 
requirements within a set of system constraints. These performance requirements are defined as the Figures of Merit 
(FOMs) of the system, and considerable research has focused on how to represent these FOMs in algorithmic form. 
The applied set of FOMs should reflect the overall goal of the selection process. The following list of general FOM 
categories was selected after reviewing the available literature and research. 

■ Observability — This category considers how well the sensor suite will provide information about the given 
system process, which parameters are directly observed, and which parameters can be inferred. 

■ Sensor Reliability/Sensor Fault Robustness — This category addresses sensor reliabilities and how sensor 
availability impacts the overall sensor suite performance. 

■ Fault Detectability/Fault Discriminability — This category specifically addresses whether the sensor suite can 
detect and discriminate system failures. 

■ Cost — This category can include development, purchase and maintenance costs for the sensors as well as 
resource and communication costs. 

These FOMs will be discussed in greater detail in Section III, where we will identify specific methodologies and 
highlight examples of research that define them. 

B. Optimization Processes 

Analysis has shown that general sensor selection problems addressing diagnosability, or observability, are NP- 
complete (Non-deterministic Polynomial time) and are therefore computationally intractable . 1 NP-Complete 
terminology refers to how the problem’s computation time grows with the number of variables. These types of 
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problems require fast approximate search solutions in order to generate acceptable results in reasonable time. Where 


possible, brute force or exhaustive search methods are preferable as they are trivial to implement, systematically 
evaluate each candidate and will always find a solution if it exists. But as systems become more complex, the 
processing time involved for these methods grows prohibitive. Therefore methods have been developed which 
attempt to refine the searches through determination of potential candidates in order to find the optimal or near- 
optimal solution. Regardless of the selection strategy, sensor selection is based on a set of criteria called the 
objective function which is an algorithmic representation of established FOMs and system constraints. 

Many techniques have been developed for general optimized solution searches. These techniques range from 
applying constraints on the objective function to streamline the optimization process 2 to applying advanced artificial 
analysis techniques such as genetic algorithms and simulated annealing algorithms. 3 Unique optimization techniques 
have been proposed such as particle swarm optimization 4 and cutting and surrogate constraint analysis. 5 The point of 
this section is not to review each search solution or optimization technique but to identify the variety of algorithms 
available. Optimization techniques are well documented. 6 Table 1 reports the algorithms encountered in this 
literature review and references that utilized them. 


Table 1 Sensor Selection Optimization Techniques 

Encountered and Associated References 

Optimization technique Researcher 


Constraint-Based Search 
Exhaustive/Brute Force Search 

Genetic Algorithms 

Particle Swarm Optimization 
Graph Theory — Spanning Trees 
and Cutsets 

Cutting and Surrogate Constraint 
Analysis 

Mixed Linear Integer 
Programming 

Simulated Annealing Algorithm 


Narasimhan , 7 Mushini 8 
Debouk , 2 Mushini , 8 
Narasimhan , 7 Madron , 7 Worden 3 
Musulin , 10 Mushini , 8 
Spanache , 11 Sen , 12 Santi , 13 
Worden 3 
Zhang 14 

Ali , 1516 Bagajewicz 17,18 
Azam 19 

Bagajewicz, 20 ‘ 2 “ Chmielewski 23 
Worden 3 


III. Historical Review of Sensor Selection Processes 

The background for the selection of the FOMs presented in Section II is described below. These metrics are 
based on qualitative as well as quantitative methods and derived from a number of disciplines such as controls 
theory, estimation theory, data reliability and failure analysis. The subsequent subsections will review some of the 
research development that has been done in each of the first three categories: observability, sensor reliability/sensor 


6 

American Institute of Aeronautics and Astronautics 



fault robustness and fault detectability/fault discrimination. The fourth category, cost, is a general penalty-driven 
FOM grouping and is very straightforward conceptually and in implementation. There is overlap between the 
arbitrarily defined categories, and the final sensor selection FOM may be a combination of performance and cost 
metrics from any or all of them. 

A. Observability 

Of ultimate importance for any sensor suite is the concept of observability. Basically, observability is the 
capacity of the sensor network to provide information about the state parameters deemed important for performance 
monitoring, health assessment, and/or control of the system. This information may be provided as direct 
measurements of the system parameters or as in data reconciliation, the reconstruction of unobservable system 
parameters based upon observable or sensed ones. Approaches vary in the determination of the observability of the 
sensor network. Some treat observability as a true or false property, while others devised more sophisticated 
schemes to display the degree of observability. 

Some approaches used graph-based analysis for which the structural information of a system is represented by a 
directed graph, called a digraph. Observability is then based on analysis of cutsets and cycles generated from the 
digraph. Kretsovalis 24 proposed two graphically-based algorithms for classification of observable and redundant 
variables. Luong 25 also used graph-based analysis to establish an incidence matrix, relating the process relationships 
to the state variables qualitatively. Decomposition of this incidence matrix using a Gauss- Jordan elimination process 
identified whether an unmeasured variable was observable and whether a measured variable was redundant. Using 
Graph Theory and cutsets, Bagajewicz 18 defined the degree of observability and degree of redundancy for a sensor 
network. For an unmeasured variable, the degree of observability is the maximum number of sensors that can be 
eliminated with the variable still observable; for measured variables, the degree of redundancy is the maximum 
number of sensors that can be eliminated and the measurement remains redundant. These concepts were combined 
to define a degree to which system variables can be estimated by the measurement network. 

Other approaches attempted to define the degree of observability by analyzing the numerical relationships of the 
system process. By taking advantage of linear state space system theory, observability and controllability can be 
represented in matrix form. The system is observable or controllable if the observability or controllability matrices 
are of full rank or nonsingular. These represent binary conditions as to whether the system is observable/controllable 
or not. Dochain 26 considered both the rank and condition number of the observability matrix for an optimal sensor 
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placement within a bioreactor system. The Gramian of both the observability and controllability matrices have been 
used by researchers 27 28 to develop scalar metrics that indicated the degree of each respective property. 

A large amount of research has been devoted to control design and actuator/sensor placement locations, 
particularly in structural types of problems. Papadopoulos 29 investigated a number of quantitative techniques used to 
optimize structural sensor placement to select sensor locations that would observe the maximum amount of system 
information, in this case energy. Each technique attempted to characterize the amount of energy the sensors would 
observe and eliminate potential sensor locations that had minimal observability. Hac 30 proposed a methodology that 
incorporated eigenvalues from both the observability and controllability matrices to select sensor locations for 
control of flexible structures. 

Several researchers used the predictive error of a given sensor suite relative to a specific reconciliation technique 
as the performance metric. After performing matrix decomposition on the linear system model coefficients, Madron 9 
proposed a simple local optimization of this baseline set of sensors by computing the prediction precision of 
unmeasured required state parameters. The baseline sensor suite performance was compared to suite variations 
where a single sensor was removed and the objective function recomputed. Chmielewski 23 developed the framework 
for the justification and incorporation of the error covariance matrices for sensor suite performance properties and 
demonstrated how that could be optimized with other suite performance constraints using mixed linear integer 
programming techniques. Musulin 10 proposed a sensor selection process that maximized a Kalman Filter 
performance, via the error covariance matrix diagonal terms. Musulin pointed out that although the objective is to 
maximize the performance of the filter for all variables, maximizing the performance for one particular variable may 
not be compatible with maximizing it for others, resulting in a conflict. Therefore they proposed two possible 
approaches; one method would select the variable with the lowest performance value. The second approach would 
evaluate the performance of each variable relative to an “ideal” sensor suite performance, combining into an overall 
performance metric. Mushini 8 used a similar approach of maximizing a Kalman Filter performance for an aircraft 
gas turbine engine. The metric was defined as the summation of the error covariance matrix diagonal terms 
normalized to the reference or ideal system performance. 
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B. Sensor Reliability/Sensor Fault Robustness 

Regardless of the purpose of the sensor network, consideration must be made for the availability of the sensor 
signal information when it is required. Sensor faults can be common and the potential for interruption of information 
flow and how that is to be handled is an important consideration in the sensor network design analysis. 

Luong 25 investigated sensor network reliability from a control perspective that the reliability of an 
instrumentation system is the probability that information required for control is available through measurements or 
deduction during some time period. Based on that idea, overall sensor network reliability metric was defined as an 
expression of the individual sensor reliabilities. Sensor reliabilities were expressed as a function of time; therefore, 
the integration of the time profile for each sensor network was used as a quantitative comparison metric between 
competing networks. 

Ali 16 introduced the concept of reliability of the estimation of variables, which relates the sensor reliability with 
its availability to provide system state estimations. The focus of the research was for a given sensor network, how 
many different ways can a variable be estimated and if sensors fail, can a variable still be estimated. The network 
objective function was defined as the minimum product of the input sensor reliabilities for each estimated parameter. 
The initial processing routine involved graph theory concepts of chords, cutsets, and spanning trees to establish sets 
of estimated variables and their associated sensors used to observe the variable. This objective function was 
extended to include hardware redundancy (measuring a variable using more than one sensor) and spatial redundancy 
(more than one way of estimating a variable ). 16 Within the graph theory algorithm, the sensor network can be 
optimized with respect to only a single criterion; therefore, this objective function was adapted for processing with 
genetic algorithms 12 and mixed-integer nonlinear programming (MINLP ) 20 which enabled it to be evaluated with 
other performance metrics, such as sensor costs. 

Another consideration of the sensor network is the performance in the presence of sensor faults. Sensor networks 
must be robust to sensor faults, both in being able to perform adequately with a faulty sensor within the network and 
being able to continue performance adequately if a sensor is removed. Bagajewicz 17 proposed a set of performance 
metrics for a sensor suite that defined the ability of the sensor network to detect sensor faults (error detectability) 
and handle sensor faults relative to reconciliation precision performance (availability — precision after failed sensor 
removed, and resilience — precision in the presence of a sensor fault). Bagajewicz 21 extended this research from the 
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graphical tree-search-solution procedure to the explicit MINLP formulation and also extended the theory for explicit 
consideration of hardware redundancy. 

C. Fault Detectability and Fault Discriminability 

For fault diagnosis, the ability of the sensor network to detect and discriminate failure modes and anomalous 
conditions is of prime importance. This is an extension of the observability analysis where specific state variables or 
groups of state variables and their response are indicative of health conditions. The performance metric must define 
the sensor network’s ability to distinguish these fault conditions from nominal operation (fault detection) and 
distinguish these fault conditions from each other (fault discrimination). 

Significant research focuses on the qualitative or heuristic representation of the fault progression process. 
Research conducted by Raghuraj, 31 Bhushan, 32 and Bagajewicz 22 utilized directed or sign-directed graphs to 
generate fault signatures. A directed graph of a hypothetical process with faults, F„ connecting to sensors, Sj, 
indicating that the fault will affect the reading of the corresponding sensor, is displayed in Figure 1. These signatures 
provided only an indication that the sensor should respond to the particular fault condition, but the actual signature, 
magnitude or rate, was not utilized. Spanache 11 used the physical constraints of the system to develop a set of 
analytical redundancy relationships between the measured and unmeasured variables, and qualitatively established 
an influence matrix between system components and possible sensor parameters. Fault signatures were assigned to a 
failed component without regard to specific component failure modes. Flence a fault signature matrix was generated 
between the component faults and the influenced analytical redundancy relationships. Narasimhan 7 used temporal 
causal graphs, and Yan 33 used a CAD system model to develop qualitative fault signatures. Representing the fault 
signature response qualitatively avoids the cost of simulating the fault in hardware and/or software, and often this is 
the only level of fault information available during the early development of a system. 

In an effort to incorporate more fault information, Zhang 14 proposed a quantified directed graph (QDG) to model 
the fault propagation. For the QDG, each node represented a potential sensor with an associated signal-to-noise 
ratio, and each arc contained the fault propagation gain and direction between the sensors, along with fault 
propagation time. A sensor detectability value was computed for each sensor in the suite for a pre -defined set of 
faults. Fault detectability was averaged over the available sensor suite and evaluated against an assigned lower 
bound. Fault resolution was still treated somewhat qualitatively by comparing sets of sensors influenced by each 
fault. 


10 

American Institute of Aeronautics and Astronautics 



Azam 19 proposed a method that used a system model and reliability data to establish cause/effect dependencies 
between the faults of interest and the effects of faults on observable system parameters. The detection and false 
alarm probabilities associated with the failure source and observable discrepancy were utilized. A general multiple 
fault diagnosis algorithm was used to search for the most likely candidate fault that explained the set of observed 
discrepancies. Three diagnostic performance metrics were established for each potential sensor based on the 
decision probability error from a number of simulation runs. Selection evaluation was performed for candidate 
sensor networks by summing these performance metrics over the sensor suite and incorporating cost constraints. 



Fig. 1 Directed graph of a hypothetical system 
displaying the propagation of possible faults (Fj) 
through the sensor (Si) network. 
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Fig. 2 Nondimensional fault trajectory space for a 
two sensors, Si and S 2 . 

S anti 11 utilized a high fidelity dynamic model of a rocket propulsion system to generate a simplified linear 
model. Health parameters were identified within the linear model that represented specific fault conditions. This 
research focused on mean-shift faults in the system. An inverse diagnostic model was developed which predicted the 
change in health parameters based on sensor inputs. Fault trajectories were generated and metrics were established 
for detection timeliness and fault discrimination. Fault detection required sufficient measurement deviation to 
discriminate a fault from normal operation variance. Detection thresholds were established based on the historical 
information of measurement variance, anticipated process variance and defined false alarm constraints of the 
system. Figure 2 illustrates a 2-D representation of the fault trajectory space where the axes represent sensor change 
from a nominal state (the origin) in increments of the sensor nominal standard deviation (a). The diagnostic 
performance metrics were based on the distance measures between fault trajectories for fault discrimination and 
inferred time to detection threshold areas for detection timeliness. In addition, fault risk reduction measures were 
incorporated into the evaluation process to allow weighting of individual failure modes based on criticality and 
occurrence rates. 


IV. Proposed Sensor Selection Approach 

While the design of aerospace systems/subsys terns does not necessarily require an analytical methodology to 
select the location and quantity of sensors, sensor placements based solely on expert “best guess” can result in 
nonuniformity across the system and can be unverifiable especially for competing objectives such as maximizing 
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safety and reducing costs. Therefore, a systematic approach based on accepted techniques and metrics is desired that 
allows for sensor network trades and consistent evaluation of metrics across the system. We direct our focus toward 
ensuring that the health assessment requirements are met by the sensor network; in that respect, several key 
questions must be addressed through evaluation of the sensor networks: 

■ Can the network provide a sufficient level of detectability for the failure modes of concern? Is the sensor 
network able to detect each failure and how early after the fault is initiated? 

■ Can the network discriminate between the various system failure modes? Failure discrimination need only be 
performed to the level where remediation strategies vary. 

■ Can the network perform the diagnosability within the cost constraints established by the systems? This 
would include development, installation and maintenance costs, processing costs, power requirements, and 
weight. 

■ How robust is the network in the presence of sensor failures? What sensors require redundancy? Are there 
common mode failures among the sensor network that could be overcome? 

In addition, it is important to utilize the maximum level of fidelity available in establishing the justification for 
sensor selection; therefore, quantitative approaches are preferred over qualitative approaches, but the methodology 
should be flexible enough to incorporate the best system design information available. The methodology selected 
should also focus on known failure modes and allow the user to incorporate weighting factors that represent both 
fault criticality and sensor costs. The methodology should incorporate other metrics, such as sensor reliability and 
sensor fault robustness, as needed. 

The approach that we have selected will advance the method described in Ref. 13, Systematic Sensor Selection 
Strategy (S4). This approach to sensor selection provides an architecture that fulfills the objectives outlined above, 
and a specific example of the S4 methodology to a propulsion system is provided. Although the demonstration 
utilizes a specific diagnostic engine, this methodology may be applicable to a wide range of diagnostic techniques. 
In addition, the architecture of this methodology should allow the implementation of other evaluation metrics and 
optimization processes liked those reviewed in Sections II and III. The overall objectives and architecture for the 
proposed sensor selection approach are given below. 
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A. Objectives 

The goal of the diagnostic sensor selection methodology is to provide a systematic, justifiable, creditable 
evaluation of the available sensor suite, relative to the diagnostic requirements. The methodology should be 
applicable to multiple types of systems: propulsion, electrical, hydraulic, etc. Each application and implementation 
may require unique metric formulations, application-specific diagnostic models, and tailored optimization selection 
strategies, therefore the systematic approach should be flexible and modular. 

The methodology should be applicable across various phases of operation for the system being applied. Often the 
same system may have distinct modes of operation where diagnostic requirements may indicate unique sensor 
networks. For example, the faults and corresponding sensors required to monitor/detect/diagnose those faults at one 
operating mode of the system (e.g., engine firing) may not be the same required at another (e.g., quiescence or 
dormant checkout phase). 

The methodology should apply throughout the design phase, regardless of the fidelity of simulations and domain 
information. As noted by both Santi L ' and Musulin 10 , the fidelity of the model utilized will impact the performance, 
health assessment or data reconciliation, and therefore the sensor suite selection process. Chmielewski 23 proposed 
that the quality of the reconciled values will depend more on the sensor configuration than the particular 
reconciliation algorithm employed. Therefore, the inputs, simulations, and fault and sensor information, based on 
“best available information” are required at each stage of the design process. Either the design analysis tool should 
demonstrate applicability throughout or define where in the process it is relevant. 

Finally, the sensor network performance depends upon both the sensor suite available and the analysis technique 
applied. An optimum sensor network established for a specific technique may not be appropriate for another 
technique. Care must be taken in generalizing the performance results across analysis functions to ensure 
appropriate solutions. The sensor selection architecture should allow different diagnostic assessment modules, 
optimization techniques and performance metrics to be utilized. 

To that end we are proposing an architecture that will enable the following capabilities: 

■ A toolbox of representative cost and performance metrics, allowing the user to specify how those metrics will 
be defined and combined. 

■ Weighting capability at the component fault, sensor measurement and performance criteria metric levels to 
allow for importance to be established by the user. 
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■ Established interfaces between modules to allow implementation of diagnostic modules and optimization 
techniques. A plethora of diagnostic algorithms, optimization search algorithms, and system simulations exist 
that could be utilized for this methodology. Standardization and definitions of interfaces within the 
framework will enable the inclusion of these other modules. 

B. Architecture Formulation 

Figure 3 illustrates the sensor selection architecture proposed by Santi 13 and the one that the authors have 
selected as the framework for further development and verification. The architecture can be segmented into three 
groups: Knowledge Base, Iterative Down-Select Process, and Final Selection. A general discussion and the 
implementation of each section relative to the S4 methodology are provided. A more detailed description of this 
process, with examples of how to formulate the problem is provided by Sowers 34 . 
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Fig. 3 Proposed diagnostic sensor selection architecture. 


C. Knowledge Base 

The knowledge base includes the health-related information and the system simulation. The health-related 
information about the host system is collected throughout the design and development. If certain data on the host 
system is not available in the early stage of design and development, experience from similar systems can provide a 
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basis from which to define and collect pertinent health assessment information. System simulations also constitute a 
key role of the knowledge base and can range in fidelity depending on the knowledge of the host system. 

Health-related information is extracted from domain experts, manufacturer reports, Failure Modes and Effects 
Analysis (FMEA) and hazard analysis studies, and will help define fault signatures, fault progressions, 
component/sensor reliabilities and sensor characteristics (e.g., noise). This data supports all aspects of the system 
architecture including the development of system simulations. 

System simulations generate data sets required to develop the system diagnostic models and evaluate the 
performance of the sensor suites. To evaluate diagnostic performance, the simulation must be capable of providing 
information on nominal and failure scenarios. This module may be a collection of simulations of varying fidelities 
and applicable at varying stages of the system operation. When available and appropriate, real test or flight data 
could be provided from this module. 

Data are usually conditioned before use in the sensor selection process. Data conditioning provides numerical 
stability for manipulation of the observation vectors that contain a variety of measurement types. It also assists the 
user by transforming the data into a form that is easier to comprehend from a health monitoring perspective. For the 
S4 implementation, this conditioning is done by subtracting the baseline expected value from the measured sensor 
value and dividing by the in-run measurement uncertainty. 


conditioned, i 


(y. 


observed ,i ybaseline. 


ine.i ) 


a 


yi 


(1) 


In-run measurement uncertainty is largely due to system and/or signal noise and is assumed to be a Gaussian 
distribution about a mean value, with a standard deviation, a v . For sensor measurement, the conditioned value 
therefore represents the number of uncertainty intervals the signal varies for a given fault scenario. 

Within the system simulations there are parameters that reflect health status of the host system and are identified 
as hardware fault parameters. The health parameter values are conditioned using the relation: 


V conditioned, j 


observed, j ^baseline j ) 


^ baseline j 


(2) 
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The conditioned health parameter value represents the percentage shift from the nominal baseline value for a given 
fault scenario. The value of xj in equation (2) can be directly obtained from the system simulation. But if the actual 

hardware is used instead, this parameter can not be directly measured and it would need to be estimated using the 
sensor values. While the notation will be removed for simplification, the sensor measurements and hardware fault 
parameters to be used in the formulations presented in the remainder of this document are to be conditioned. 

To evaluate the performance of the sensor suites in the selection process, fault test scenario data sets are created. 
To determine the fault magnitude to be used in each fault test scenario, a procedure such as a fault detectability 
study can be employed. In a fault detectability study, fault data are generated across a range of pertinent fault 
magnitudes. The range usually extends from a nominal condition to the maximum fault magnitude for which data 
can be obtained or is practical. Fault detection criteria are defined usually from application specific requirements, 
such as missed detection allowance, false alarms allowed and/or detection timeliness. Data are then analyzed to 
determine the minimum fault magnitude that satisfies the fault detection criteria. Typically, the minimum fault 
magnitude scenario is used as a fault test case for sensor suite evaluation. 

D. Iterative Down-Select Process 

The iterative down-select process is a procedure to select a group of near-optimal sensor suites for health 
assessment. The process involves sequentially generating and evaluating a collection of candidate sensor suites, 
eventually converging towards a more optimum sensor suite based on the defined performance metric or objective 
function. The process includes a system diagnostic model, a sensor suite merit algorithm, and a down-select 
algorithm. 

1. System Diagnostic Model 

The performance of the diagnostic system is measured for each candidate sensor suite at each relevant 
operational mode. The measures of performance are provided to the sensor suite merit algorithm. A key feature for 
this model is that it must be complete, and capable of providing diagnostic analysis for all or any subset of the 
possible sensors. A diagnostic model that must be redefined for each new sensor suite would not be appropriate for 
this architecture. 

The system diagnostic module employed should be representative of the actual operational diagnostic system 
since the down-select process will optimize the sensor suite based upon that particular diagnostic process. The 
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diagnostic system commonly utilized by the authors is the inverse model. An inverse model contains a parameter 
optimization technique that performs model inversion to determine the hardware fault parameter type and magnitude 
that best recovers the observed measurement values. By monitoring hardware fault parameters, abnormal 
observations can be traced directly to the anomalous component and provide a means for fault detection and 
isolation. The accuracy of the hardware fault parameter reconstruction reflects the suitability of the sensor suite 
under evaluation. 

The inverse model used assumes measurement parameters are piecewise linear functions of hardware fault 
parameters. For each hardware fault parameter, Xj , end points of the linear subintervals, p. are referred to as 

interpolation points. An ordered sequence of interpolation points x p A x <x Pj 2 <---<Xj JS are selected to establish a 

linear function within each subinterval. Interpolation points do not need to be evenly spaced, as regions with 
increased non-linearity may contain more interpolation points, while more linear regions may contain less. 

For each hardware fault scenario, k. the value of sensor i at each hardware fault parameter^ interpolation point p jq 
is provided by the system simulation as indicated by the relation below 


Pjt _ \fi(Xj j9 ) for k = j 

y v i 

[0 for k * j 

where / = 1, 2, . . . m ; j = 1,2,... n; k = l,2,...n; q = 1,2,... s; 


( 3 ) 


To implement the piecewise linear approximation, the hardware fault parameter influences on the sensor 


measurements over the subintervals 


~PM y.Pi(q+ 1 ) 

x j ’ X j 


were represented by constant influence coefficients A-' 1 ' 1 . The 


solution of a targeted fault condition, x T : , the change in any sensor i from its nominal value was then approximated 


by the linear relation: 


= fi{x l , 
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This linear relation uses the appropriate hardware fault parameter interpolation points x 1 ’ 1 ' 1 such that 
x T j g [ x j J<< ,Xj Aq+l) ] with the appropriate subinterval influences defined by the relation 



P](q+l) _ Pjq 

y ij y ij 


Pj(q+V) 

X j 



(5) 


A modified Levenberg-Marquardt nonlinear optimization algorithm 35 is used to solve the following nonlinear 
optimization problem in order to predict the closest match for each of the n hardware fault parameter values for each 
set of observations. In other words, for each hardware fault parameter the algorithm minimizes the absolute arrow, 
y,- , between the observed sensor values y, and the estimated sensor values y, . 



( m \ 


^ m ^ 

d = min 


- min 

i>-*i 


Ki = 1 ) 


\i = i 7 


by selection of Xj j = 1,2, n 


(6) 


This is done by using the Levenberg-Marquardt optimization algorithm 4 to adjust the estimated health parameter x ,■ 
in order for the observed sensor measurements to closely match the estimated sensor values. The j) ( - values are 
computed from the approximate model relation given in equation 4. Figure 4 illustrates the optimized results from 
the inverse model for a simple three sensor, and three health parameter case. The optimized distance, d,. represents 
the closest point along each fault trajectory, Xj , to the current observation, y observed ■ 
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Fig. 4 Example illustration of optimized distance 
results from the Levenberg-Marquardt algorithm. 

For proper fault detection and identification, the optimum point along the health parameter trajectory should be 
beyond the established detection spheroid. The distance for the targeted fault (the fault represented by the 
observation) should be small, indicating proper identification. Also the distance for the remaining faults should be 
large indicating amount of fault isolation available. It is these three factors that are available from the inverse model 
to characterize the diagnostic capabilities of the sensor suite. 

2. Sensor Suite Merit Algorithm 

The merit function assessment procedure employed by the authors can be summarized as follows. First a sensor 
suite is selected from the grouping of candidates and is evaluated against each of the fault test cases. In a fault test 
case evaluation, the ability of the sensor suite to detect and discriminate the correct fault condition is determined by 
solving for every potential fault condition using the inverse model as the diagnostic algorithm. Fault scenario results 
are then used to “score” the sensor suite health monitoring performance. The process repeats until every candidate 
sensor suite in the group is appraised. 

With the basic procedure given above, a detailed merit algorithm procedure is offered. A candidate sensor suite 
is selected and only the measurements represented within the sensor suite are utilized. An initial detection check is 
made to ensure that the suite of normalized values is above a minimum detection level. If detection criteria are not 
met then the sensor suite is classified as insensitive. Any sensor suite insensitive to the fault test case is not 
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evaluated by the inverse model diagnostic algorithm as it does not have a fault signature significant enough for fault 
discrimination. If the sensor suite is sensitive to the fault scenario, it is evaluated by the inverse model diagnostic 
algorithm. During evaluation, the value of each health parameter is recovered individually along with the estimated 
sensor values at the recovered health parameter value. Because an iterative optimization process is employed, there 
will not be an exact match of the input sensor values to the sensor values recovered by the inverse model. By 
computing this difference, the inverse model provides the residual sensor values of the recovered fault case. 

The recovered sensor values are compared to the actual input sensor values. Only the residuals beyond a 
normalized threshold, T h are combined into a root-mean squared value for the selected sensor suite. The normalized 
threshold is used to define the boundary of “good” recovered measurement data that match the input fault test case 
data to account for the iterative optimization procedure. 


D 


residual, j 


m 


m 

7, (f resid_ err,i ) 


( 7 ) 


(=i 


where 


S resid err.i 


1 \y residual, i\ 7/ if \y residual, i | > 7/ 

0 if \y residual, i | — 7 / 


The residual measurement agreement value is computed for each fault test case for each of the possible fault 
scenarios and combined into an overall fault discrimination metric for the sensor suite. The final agreement value 
should reflect a preference for small residuals when the fault input type matches the recovered fault type and large 
residuals when the fault types do not match. 


n 

z k =y«7) 


residual, j 


7=1 
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If fault evaluated ^ fault true _ input 
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The merit value is constructed through the multiplication of metrics characterizing the desirability of the sensor 
suit by the sum of the fault detection and discrimination performance metrics for each fault case. The merit value of 
the sensor suite is computed as, 


n 

Merit = UP^C k W k Z k (9) 

k = 1 

The weighting variables C k and W k allow the user to prioritize the fault test cases by importance, occurrence or 
criticality. 

The utility weighting factor, U, is a metric that incorporates a weighting term for the individual sensors in the 
measurement set. This factor is simply an average of pre-assigned values for each of the sensors. This utility term 
could incorporate factors such as cost, weight, power requirements, etc. The utility term is calculated as, 


U = ^ — 


(10) 


The penalty term, P, is a factor that punishes a sensor suite due to a deviation from a desired characteristic. It is 
computed by, 


P = 


K 


K + ( Wpenaltv Ndesired — 


(11) 


This formulation for the penalty term includes two design parameters, K and W pe „ a iiy, which the user specifies. This 
penalty term decreases from an optimal value of 1.0 as the number of sensors contained within the candidate suite, 
N actual? deviates from the desired number of sensors, Ndesired, specified by the user. 

3. Down-Select Algorithm 

The down-select algorithm is a search algorithm that utilizes an optimization technique employed to select sensor 
suites that will allow progression toward an optimal or near-optimal solution. Due to the large search space of the 
candidate sensor suites, the use of an optimized search algorithm allows for a timely generation of candidate sensor 
suites. The down-select algorithm receives inputs from the health-related information and sensor suite merit 
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algorithm, and generates a collection of new candidate sensor suites that should “evolve” toward the optimum 
solution. 


Parentl = 

Parent2 = 

Childl = 

Child2 = 

Fig. 5. Single point crossover. 

A genetic algorithm (GA) is used by the authors as the down-select algorithm. Encoding the sensor selection 
problem in a GA format is a straightforward process. A population consists of a collection of candidate sensor 
suites. Each individual member of the population is referred to as a chromosome and represents a single candidate 
sensor suite. Each chromosome is composed of bits called genes associated with individual candidate sensors. 
Binary encoding of the gene (1 or 0) indicates if the particular sensor associated with the gene is in the sensor suite 
represented by the chromosome. A chromosome mask is employed to force either selection or omission of particular 
sensors as desired. Each chromosome in the input population has an assigned merit value received as input from the 
merit algorithm. To choose output sensor suites passed to the next generation, roulette-wheel selection is employed 36 
whereby sensor suites with higher relative merit values have an increased probability of being selected as parents. 
Once parent suites are selected, a determination is made if crossover occurs. If it does, single point crossover is 
employed to construct offspring sensor suites. The crossover process is an operator whereby a gene sequence is 
copied from one chromosome to another. The crossover point within the parent chromosomes is selected randomly 
and genes are copied from the parental chromosomes to the child chromosomes as depicted in Figure 5. If crossover 
does not occur, the parents are simply copied to the next generation. Elitism is used to advance a user-defined 
number of best sensor suites to the next generation without modification. Mutation or random bit-flipping is 
employed during parent-to-child gene copying for its diversifying effects. As generations progress, the population 
will increasingly converge to a group of similar sensor suites providing good diagnostic capability. Increasing the 
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probability of mutation with each generation aids in increasing the diversity of the population. Preserving diversity 
assists the population in evolving away from local optima in the solution space. 

The basic steps of the GA employed are outlined below. 

1. Initialize — Input the initial population of sensors from the knowledge base and make it the current 
generation. The initial population is randomly generated. 

2. Merit Function — Input the merit algorithm assigned value for each sensor suite in the current generation. 

3. Elitism — Automatically advance a given number of sensor suites with the highest merit value to the next 
generation. 

4. Selection — Select two sensor suites using roulette -wheel selection. 

5. Crossover — Determine if crossover occurs. 

A. No Crossover — Both parents advance to the next generation without modification. 

B. Crossover — Apply single point crossover with the possibility of mutation as each gene is copied. 

6. Repeat — Return to Step 4 until next generation is populated. 

7. Reset — Set the current generation equal to the newly formed next generation. 

8. Output — Send current generation of sensor suites to the system diagnostic model to reinitiate the merit 
assignment process. 

9. Repeat — Return to Step 2 until target number of generations is reached. 

E. Statistical Evaluation Algorithm 

The statistical evaluation algorithm is intended as an optional final robustness test of each candidate sensor suite. 
When constructing the iterative loop of the down select process, often there is a trade between simulation fidelity 
and speed. Therefore, a more rigorous evaluation is employed when a finite collection of “optimum” measurement 
sets has been determined. The statistical evaluation algorithm replicates portions of the down select process but with 
additional uncertainty effects such as sensor and system noise characteristics, and variations in fault dynamics. . The 
final sensor suites can be evaluated on their diagnostic performance using higher fidelity simulations. Each sensor 
suite is evaluated with the same system diagnostic model and the sensor suite merit algorithm used to initially select 
the suites in the Iterative Down-Select Process. 
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F. Formulation Conclusion 

The result of this selection process is a set of viable candidates that the designer can further assess and carry 
forward in the system design process. Each suite will have metrics measured against its performance to enable direct 
comparisons. This approach can demonstrate the impact of individual sensor measurements or attempt to quantify 
relative benefit between sensor networks. Trade studies can be conducted to compare optional sensor networks and 
identify health monitoring gaps in the system’s measurement design. Most importantly, the sensor network designer 
will have a valuable tool with which he can defend the selection made. 

V. Design Example 

To demonstrate the procedure for creating and employing the S4 framework, an example application is 
presented. It should be noted that some of the steps illustrated in the subsequent sections of the example can be 
specific to the diagnostic module selected. For example, selection of another diagnostic approach may change the 
diagnostic module preprocessing of data and the computation of the performance metric. 

A. Host System 

The host system selected for this demonstration was the Space Shuttle Main Engine (SSME). The scope of the 
host system was restricted and only a few fault scenarios were considered. Although sensor selection methodologies 
are normally applied at the system level and are more complete, limitations were imposed to simplify the 
demonstration. 

As displayed in the simplified flow schematic of the SSME in Figure 6, there are three targeted fault conditions 
(shown by the arrows) and ten candidate sensors indicated by the red-filled circles. The three fault conditions of 
interest are a nozzle coolant flow leak, a high pressure oxidizer turbopump (HPOT) efficiency loss, and an oxidizer 
preburner (OPB) fuel injector resistance increase. For simplicity, the fault conditions are restricted to periods of 
steady state operation and multiple faults are not considered. The candidate sensors considered for this study are not 
currently flight measurements on the SSME. The candidate sensors are listed in Table 2. 
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OPB fuel 



Fig. 6. Simplified SSME flow path schematic for S4 methodology demonstration. 

The SSME Power Balance Model (PBM) was used as the system model. The SSME PBM is a complex engine 
model that allows a user to adjust internal engine performance parameters and generate simulated engine 
measurement data. Three internal engine health parameters were identified, each representing one of the targeted- 
fault modes as indicated in Table 3. A range was established for each targeted fault, from zero (nominal) to the 
maximum fault level, and partitioned into an arbitrary number of fault levels. For this demonstration, 9 fault levels 
were defined over the fault range. Each fault level was defined by a corresponding engine health parameter value. 
The SSME PBM was then used to generate simulated data for each of 27 (3 targeted health parameters times 9 
distinct fault levels including a nominal scenario per health parameter) reference cases. Model output variables for 
all candidate sensors listed in Table 2 were recorded. 
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Table 2 SSME Candidate Sensors for Restricted Demonstration 


PBM variable 

Candidate sensor 

hpfpdp 

High pressure fuel pump discharge pressure 

hpfpip 

High pressure fuel pump inlet pressure 

lpftps 

Low pressure fuel turbopump speed 

hpopip 

High pressure oxidizer pump inlet pressure 

hpotps 

High pressure oxidizer turbopump speed 

opbpc 

Oxidizer prebumer chamber pressure 
Oxidizer prebumer oxidizer value actuator 

opovx 

position 

Main combustion chamber oxidizer injector 

mccoip 

pressure 

opbfip 

Oxidizer prebumer fuel supply duct inlet pressure 
High pressure oxidizer turbine discharge 

hpotdt 

temperature 


Table 3 PBM Health State Parameters Defined by Fault 


Condition 

PBM variable 

Targeted-fault mode 

NozLk 

Nozzle coolant flow leak 

HPOTef 

OPBfiR 

High pressure oxidizer turbopump efficiency loss 
Oxidizer prebumer fuel injector resistance increase 


VI. Results 

The S4 demonstration was executed with perturbations to parameters in the penalty term. The intent was to 
examine the number of actual sensors in the final solution sets versus desired number of sensors selected by the user. 
The results are presented in Table 4. The first column represents the desired number of sensors, the second column 
represents the weighting penalty applied and the remaining columns indicate the sensors selected from the S4 
process. The normalizing term, K, is arbitrarily set to one. With W pena ity values of 0.0 and 0.01, the same three 
measurements, hpfpDp, hpotDT and hpopIP, were selected regardless of the desired number of sensors specified. 
This measurement set scored well due to its sensitivity to the targeted fault conditions permitting it to overcome the 
penalty term as it was applied. When the W pena ity value was raised to 0.1, the penalty term was able to drive the 
solution set to the desired number of sensors with specific exceptions. When the target number of sensors was one, 
the minimal solution set contained opbPc and hpotDT. On the other extreme, as target sensors were increase to nine 
and ten, opovX and opbFiP were never selected. Inclusion of these two measurements decreased the merit value 
more than the penalty term decreased the merit value for non-inclusion. 
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Table 4 Selected Sensors as a Function of the Designated Number of Sensors in the Measurement Suite 


N desired 

^ penalty 

hpfpdp 

hpfpip 

lpftps 

hpopip 

hpotps 
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opbfip 
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A plot of the merit value for each test scenario and a weighted sum of all test scenarios is given in Figure 7. The 
plot shows an overall optimum number of sensors to be three although the ideal number of sensors varies for each 
fault case. Beyond the ideal number with each sensor added, the merit value will progressively decrease due to a 
reduction in the ability of the inverse model to discriminate the correct fault condition. For the system used in this 
demonstration, the responses of the additional sensors are not independent to the other sensors and have the effect of 
smearing the recovered fault solution during the inverse model optimization process. 

Results from this example provide a number of design option studies for the sensor network designer. By varying 
the sensor target number in the merit function penalty term, the designer can explore the impact on the composition 
of the sensor suite as more sensors are included. Also the ability exists within the established S4 framework to force 
the inclusion or exclusion of one or more sensors from the solution set. This feature could be used to examine the 
sensor set that would be selected if one or more of the core sensors were not available. In this way the designer 
could explore information redundancy options within the sensor network. 

A more thorough analysis of the diagnostic performance results can be supported by the S4. This would provide 
the design engineers the information of what system failures are covered and what failures still need to be addressed. 
A statistical evaluation can be performed on the potential sensor suite candidates to determine their sensitivity to 
system variations (e.g. signal noise and engine-to-engine build variations). These types of studies enable the 
designer to demonstrate sensor network robustness. 
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Fig. 7 Optimal diagnostic merit value versus number 
of sensors. 

VII. Concluding Remarks 

The sensor selection process is the evaluation of sensor networks with the objective of fulfilling specified 
performance requirements within a set of system constraints. For the health assessment of aerospace systems, the 
performance requirements include fault detection, fault resolution, and a timely diagnosis. Other metrics can include 
gross error detectability and the ability to perform in the event of a sensor fault, as well as minimizing sensor costs. 
The paper includes a review of current literature from other disciplines (e.g., industrial and chemical processing 
applications) in order to assess the state of sensor selection technologies that might be applicable to aerospace 
systems. This investigation offers a prelude to the proposed architecture based on what is considered to be important 
criteria for a sensor selection approach for aerospace applications. A sensor selection architecture was proposed that 
expands on the research conducted earlier. The architecture’s flexibility will enable broader combinations of 
evaluation metrics to be incorporated. Further development and demonstration is required to advance the confidence 
of this approach with aerospace design applications. In addition, applicability to other systems beyond propulsion 
elements needs to be established. This will require the definition of a formal architecture and interfaces that will 
allow other modules to be easily integrated. Demonstrations need to be conducted to determine applicability in other 
areas. 
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